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Abstract 

This article describes initial efforts to pilot an evaluation instrument for school 
counselors. The pilot was a collaboration led by the state’s school counselor association 
involving a state department of education (DOE), local school districts, and university 
faculty members. The article begins with a brief overview of historical and contextual 
factors relevant to the creation of the instrument and the pilot project. A summary 
description of the instrument is then provided that lists individual items and 
supplementary information distributed with the protocol. Next, preliminary results are 
presented. Finally, the article concludes by discussing limitations of the study, 
implications for practitioners, and recommendations for further research. 

Keywords', school counselor evaluation; state association; department of 
education; collaboration; accountability; comprehensive school counseling program 
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School Counselor Evaluation Instrument Pilot Project: A School Counselor 

Association, Department of Education, and University Collaboration 

There remains little debate regarding the function and importance of evaluation 
within a comprehensive school counseling program (CSCP), as articulated within the 
American School Counselor Association’s (ASCA) National Model (ASCA, 2012). This 
call for accountability spans more than just evaluating CSCP programming and 
interventions (Dimmitt, Carey, & Hatch, 2007; Erford, 2015; Maras, Coleman, Gysbers, 
Herman, & Stanley, 2013; Thompson, 2012); it also incorporates evaluating school 
counselors as well (ASCA, 2012; Kaffenberger & Young, 2013). However, despite 
repeated calls for school counselors to embrace and model accountable assessment 
and evaluation (Anderson, 1994; Sink, 2009; Struder& Sommers, 2000), there remains 
a scarcity of literature regarding individual school counselor evaluation. At the writing of 
this manuscript, an EBSCO online search with the term “school counselor evaluation” 
yielded nine items, of which eight discussed school counselors and/or school 
counseling. Of those eight articles, only two were written in the 21st century. While 
expansion of the search criteria produces more yield, the trend of scarcity of articles 
remains consistent. 

One of the two articles retrieved was an interview with the director of the 
Education Trust’s National Center for Transforming School Counseling (TSC) during 
2007 (Perusse & Colbert). The article focused on the TSC, the director’s role, and the 
evolving role of the school counselor. The second article (Morris & Slaten, 2014) 
outlined a collaborative project undertaken by a multidisciplinary workgroup involving 
the state school counseling association, department of education, and other stakeholder 
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groups. Morris and Slaten (2014) describe an evaluation rubric and administration 
protocol for school counselor evaluation processes. The article is more than just 
theoretical as history, context, implementation, results, and recommendations are 
outlined, however no statistics from the research are presented. 

One potential reason for this scarcity of research may lie in the fact that whereas 
CSCP evaluation practices are influenced by school site dynamics, individual school 
counselor evaluation can fall prey to myriad systemic influences. Similar to CSCP 
program evaluation, school site systemic factors (e.g., student population 
demographics, school counselor-student ratios, school counselor full-time-equivalent 
staffing, site administration) play a key role in individual school counselor evaluation. 
Additionally, larger macro-level systemic factors such as district policies/politics, local 
union regulations, and statewide legislative pressures may exert influence over 
individual school counselor evaluation. For school counselors already struggling to 
adequately implement basic program evaluation processes within their CSCP 
(Astramovich, Coker, & Hoskins, 2005), the prospect of tackling role evaluation may 
prove too daunting. 

This article presents historical and statistical review of a pilot project related to an 
individual school counselor evaluation tool. The project was a collaboration between the 
state’s school counselor association, a state department of education (DOE), local 
school districts, and university faculty members. After providing a brief historical 
overview of the project, the authors present statistical methods and results pertaining to 
the instrument. Following discussion of these results the authors present limitations of 
the project, recommendations for future research, and (arguably most important for the 
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practicing school counselors) recommendations for continuing collaborative evaluation 
efforts. 

The authors should note espousing an ecological professional school counseling 
perspective (McMahon, Mason, Daluga-Guenther, & Ruiz, 2014) which recognizes the 
results from this research as systemic feedback framed within the time and health of the 
larger state system. Recognized as systemic elements, deliverables from such research 
(i.e., the evaluation protocol) are subject to systemic feedback. The changing landscape 
of state education politics seems to regularly challenge such instruments regardless of 
statistical merit. 

History and Context 

During 2012-2013, representatives of the Georgia state school counselor 
association (SCA) met with numerous legislators in an effort to introduce a bill that 
would define the role of the school counselor. Although unable to find a sponsor for the 
bill, SCA leadership identified several legislators supportive of the effort who suggested 
working with the state’s department of education (DOE) to implement a counselor 
performance evaluation instrument which would better define the role of the school 
counselor. Meeting with DOE leadership later that year, SCA leaders found the state 
superintendent supportive of pursuing such an evaluation. However, as DOE leadership 
did not have adequate manpower or resources for developing such an instrument, SCA 
leadership inquired about developing the instrument in collaboration with the DOE. The 
superintendent was open to the idea and requested the instrument be aligned with the 
newly developed Teacher TKES evaluation as the TKES was currently in the second 
year of its pilot and scheduled for statewide implementation the following year. 
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In the summer of 2013, a committee of two elementary school counselors, two 
middle school counselors, two high school counselors and two district level school 
counseling supervisors from the southern, central and metropolitan parts of the state 
developed the Counselor Keys to Effectiveness System (CKES). The committee utilized 
the 10-item format and much of the original language from the TKES in pursuit of 
administrator familiarity with the instrument. Additionally, school counselor performance 
evaluations based on the American School Counselor Association (ASCA) National 
Model (2012) from around the state and nation were consulted. Selected model 
evaluations were chosen based on how well they reflected the role of the school 
counselor within a data-driven, comprehensive school counseling program, and were 
aligned with the ASCA National Model. Language from the ASCA National Model was 
incorporated into each of the 10 standards of the CKES. 

In the fall of 2013, representatives from the SCA met with the superintendent and 
DOE leadership to share the newly created CKES and seek guidance regarding 
implementation of the instrument. DOE leaders involved in the TKES recommended 
several actions for completion prior to CKES implementation. These recommendations 
included: (1) facilitating a focus group to provide feedback on the instrument; (2) 
administering a survey to gauge practicing school counselor receptivity; and (3) piloting 
the instrument with administrators in various districts. 

At the subsequent SCA annual conference, 25 school counselors representing 
various levels and regions participated in a focus group. Participants were given the 
opportunity to preview, ask questions and provide feedback regarding the instrument. 
Approximately 64 percent of group participants indicated they were evaluated using a 
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counselor-specific instrument. Approximately 20 percent stated they were not evaluated 
at all. When asked whether they believed the instrument would help define the role of 
the school counselor for administrators, 23 participants agreed and two stated they 
were unsure. Comments from the group indicated that participants liked having the 
CKES aligned with both the TKES and the ASCA National Model (2012). Participants 
also expressed concern regarding support from administrators and the impact of non¬ 
counseling duties on meeting the requirements of the instrument. Approximately one 
third of participants shared their belief that training for administrators and counselors on 
using the instrument would be critical for successful implementation. Using this and 
other feedback from the focus group (e.g., item wording and concepts), revisions were 
made and a finalized version of the CKES was completed in February 2014. 

In addition to the finalized CKES instrument, SCA committee members created 
supporting documents and resources. These resources included conceptual definitions, 
links and references, contextual examples of item rating criteria, and templates based 
on the ASCA National Model (2012) (e.g., action plans, results reports). Once 
completed, the instrument and supporting documents were provided to six public 
schools from across the state that had volunteered to field test the instrument. 
Administrators and counselors involved in the field test provided feedback on the 
instrument via a survey in May 2014. No changes were recommended by those 
participating in the field test as feedback indicated the instrument was easy to use and 
helped to further define the role of the school counselor. In addition to the survey, field 
test participants were given the opportunity to provide qualitative feedback. 



8 


Representative of the positive feedback, one participant (a counselor coordinator) 
stated, 

The evaluation appears to be an effective evaluation instrument that will 1) 
capture what counselors do on a daily basis and 2) provide a sense of 
accountability for all stakeholders. At first, it seemed overwhelming, but after 
looking at each standard and digesting the details under the 4 different levels, it 
appeared very ‘cut and dry.’ 

With encouraging field test results, further data was sought to solidify the need for a 
revised instrument. 

An additional survey was administered via the statewide Career, Technical, and 
Agricultural Education Resource Network in April 2014 reaching approximately 1,128 
school counselors representing various levels from across the state. Approximately 56 
percent of respondents indicated they were evaluated with a counselor-specific 
instrument. Approximately 61 percent of respondents reported they believed an 
evaluation based on the ASCA National Model (2012) would help administration better 
understand the school counselor role. When asked if they would want to be evaluated 
with an instrument which defined the role of the school counselor within the framework 
of a comprehensive school counseling program, 74 percent indicated “yes,” 21 percent 
indicated “unsure,” and 5 percent indicated “no.” 

In May 2014, SCA committee members met with the state superintendent and 
other DOE leaders to share the findings from the focus group, instrument field test, and 
interest survey. The superintendent and DOE recommended the next step of piloting the 
instrument. As 2014-2015 would be the first year of full implementation for the new 
Teacher Evaluation (TKES), many districts previously utilizing general teacher 
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evaluations for school counselors were going to need a specific instrument for 
evaluating school counselors. As a result, the decision was made to offer districts the 
opportunity to pilot the CKES during 2014-2015. An SCA-drafted email was sent from 
the state school superintendent to all district level superintendents regarding the 
availability of the instrument and the opportunity to pilot. Interested districts were asked 
to email for more information. 

Participating districts were provided with guidelines for the pilot study, which 
helped negotiate a balance between district autonomy and research integrity. First, 
districts were allowed to choose one of three options for pilot implementation: (1) whole- 
district; (2) counselor-specific; or (3) site-specific. The whole-district implementation 
model required use of the CKES instrument for all counselors, at all sites within the 
district. The counselor-specific model allowed districts to select individual counselors 
from various sites across the district as long as a minimum 50% of district counselors 
participated. Finally, the site-specific model allowed districts to select which school sites 
(and subsequently, counselors at those sites) participated in the project. 

Districts were asked to implement use of the CKES instrument in addition to 
typical evaluative practices. Ratings were scored and recorded using the CKES 
instrument for research purposes only and carried no evaluative impact for participating 
counselors’ annual evaluation. At any time during the project, districts and participating 
counselors were allowed to withdraw from the CKES project without any adverse 
consequences. CKES researchers asked that any district choosing to withdraw from the 
project provide notification and a brief explanation. In addition to data collected from 
utilizing the CKES instrument, participants (i.e., school counselors and administrators) 
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provided both quantitative (i.e., Likert-type scale survey responses) and qualitative (i.e., 
open-ended questions) feedback at the end of the project. 

Training for participants (both administrators and school counselors) was 
provided via electronic and face-to-face workshop processes. School counselors being 
evaluated by the CKES instrument were offered training via a workshop session at the 
state association’s annual conference. Additionally, participating school counselors had 
access to the training via a webinar offered online at the state association’s webpage. 

Method 

Sample 

From the state school superintendent’s invitation email, 65 sites participated 
comprising this convenience sampling (N = 117). As previously mentioned, the CKES 
instrument was to be used strictly for research purposes only—holding no evaluative 
weight. Participating school counselors’ performance was assessed using both the 
CKES instrument and their district’s current assessment instrument. 

Participants represented 65 sites, from 9 districts across the state. Participating 
school counselors were identified as elementary level (44%), middle school level (28%), 
high school level (25%), and other (3%). This mirrored state-wide trends in terms of 
elementary level consisting of the most school counselor positions. Sample distribution 
across districts also mirrored state-wide trends to some extent as the majority of 
participants identified as working in the county with the highest number school 
counselor positions. 

Both participating administrators and school counselors were invited to complete 
online perceptual surveys regarding CKES training, the CKES instrument, and the 
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assessment process. The administrator survey sample was N = 24 while the school 
counselor survey sample yielded N = 40. 

Procedure 

Whichever of the three options participating districts chose for CKES 
implementation (i.e., whole-district, counselor-specific, or site-specific), districts 
administered the CKES using regular evaluative processes and procedures during the 
2014-15 academic year. Participants (i.e., district leadership, administrators, and school 
counselors) were provided with a CKES administration guide/resource. The guide was a 
document containing the CKES instrument in digital interactive “form” format (i.e., 

Adobe .pdf form) and approximately 30 pages of additional resources and templates for 
use by both administrators and school counselors. Specifically, supplemental resources 
provided included: 

• Introduction and purpose statements 

• Detailed explanation of each standard of the instrument providing guiding criteria 
for each standard’s ratings (i.e., exemplary or ineffective) and sample indicators 
or evidence for each standard 

• Glossary of terms 

• References 

• CSCP curriculum crosswalk worksheet 

• Classroom guidance lesson plan template 

• CSCP annual site administration agreement (ASCA, 2012) 

• SMART goal worksheet 

• Various action plan templates (Kaffenberger & Young, 2013) 

• Use of time assessment tool (ASCA, 2012) 

• Various results reporting templates (Kaffenberger & Young, 2013), and 

• CSCP program assessment tool (ASCA, 2012). 
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Completed protocols were collected and secured by the state school counselor 
association. Towards the end of the 2014-15 academic year participants were contacted 
by email inviting them to complete the online perception surveys. 

Instrumentation 

CKES Instrument. As previously mentioned, effort was taken to intentionally 
align CKES performance standards with both the established state-sponsored TKES 
formatting, and established components of the ASCA National Model (ASCA, 2012). 

The CKES instrument is comprised of 10 performance standards which, when 
aggregated, yield an overall rating score. For each performance standard, the evaluator 
is asked to rate the school counselor using a Likert-type scale with the following range: 

0 ( ineffective ), 1 (needs development), 2 ( proficient ), and 3 ( exemplary ). Additionally, 
each performance standard has an area for evaluator comments. Individual 
performance standard ratings are aggregated to yield a composite score. The 
composite score is then framed within one of four categories: exemplary (total score of 
27-30 with no needs development or ineffective ratings), proficient (total score of 17-26 
with no ineffective ratings), needs development (total score of 8-16), and ineffective 
(total score of 0-7). Formatting for the CKES instrument is such that the evaluator is 
provided with defining criteria for each performance standard as well as how each 
standard aligns with the ASCA National Model (see Table 1). 
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Table 1 

CKES Performance Standards 


Standard 

Definition 

1. Professional Knowledge 
(Foundation) 

The professional school counselor demonstrates an 
understanding of a comprehensive school counseling 
program by providing relevant learning experiences in the 
three domains: Academic achievement, career development 
and personal/social growth. 

2. Instructional Planning 
(Management/Foundation) 

The professional school counselor plans and develops a goal- 
driven, comprehensive school counseling program using 
curriculum and standards, resources, and data to address the 
needs of all students. 

3. Instructional Strategies 
(Delivery) 

The professional school counselor promotes student learning 
by implementing a comprehensive school counseling program 
by spending 80% of time in school counseling core 
curriculum, individual student planning, responsive services, 
and indirect student services and 20% in program planning 
and school support. 

4. Individualized Instruction 
(Delivery) 

The professional school counselor coordinates individual 
student planning and responsive services designed to meet 
student needs on an individual and/or small group basis. 

5. Data Collection (Accountability/ 
Management) 

The professional school counselor uses a variety of strategies 
and instruments to collect student data in order to guide 
appropriate counseling interventions and programs. 

6. Data Evaluation (Accountability/ 
Management) 

The professional school counselor evaluates student data and 
the effectiveness of the counseling core curriculum, small 
groups, and closing-the-gap data in order to assess and plan 
the school counseling program and shares program results 
with stakeholders. 

7. Positive Learning Environment 
(Foundation) 

The professional school counselor promotes a safe, positive 
learning environment which is inclusive of ALL students 
(including but not limited to race, color, religion, gender, 
national origin or disability) and advocates for student needs 
in order to reach their educational goals. 

8. College & Career Readiness 
Environment 

(Delivery) 

9. Professionalism (Foundation) 

The professional school counselor creates a student-centered 
environment which promotes postsecondary planning and the 
development of soft skills. 

The professional school counselor exhibits a commitment to 
professional ethics and the mission, vision and beliefs of the 
school counseling program and participates in professional 
growth opportunities. 

10. Communication 

(Delivery/Management) 

The professional school counselor communicates effectively 
with students, parents/guardians, district and school 
personnel, and other stakeholders in a way that enhances 
student learning and improves the comprehensive school 
counseling program. 
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Administrator CKES Perception Survey. Administrators participating in the 
piloting of the CKES instrument were invited to participate in an online survey 
measuring perceptions regarding the CKES instrument and evaluation process. The 
survey consisted of 33 questions utilizing forced-choice, open-response, or Likert-type 
scale ratings for participants’ responses (see Table 2). Likert-type scale responses 
ranged from 1 (strongly disagree) to 5 (strongly agree). 


Table 2 

Administrator CKES Perception Survey Items 

Item Text Response 

1. How many years have you supervised the school counselor(s)? Forced 

2. The directions on the instrument were easy to follow. Likert 

3. The format of the instrument was easy to understand. Likert 

4. The instrument was easy to use. Likert 

5. The instrument was a reasonable length. Likert 

6. The time for administrative preparation was reasonable. Likert 

7. The terminology used in the instrument was easy to understand and follow. Likert 

8. The rating scale was fair and equitable. Likert 

9. The suggested examples of evidence were helpful in evaluating my school Likert 

counselor. 

10. If you disagree, which standard(s) could benefit from more specific examples Forced 

11. The suggested examples provided under each standard adequately allowed me to Likert 

evaluate my school counselor. 

12. Distinguishing between the exemplary and the proficient school counselor was Likert 

clear. 

13. This instrument is helpful in defining the role of the school counselor as outlined by Likert 
the American School Counselor Association. 

14. This instrument accurately reflects the role of the school counselor in my school. Likert 

15. This instrument adequately reflects the leadership role of the school counselor in Likert 

my building. 

16. This instrument changed or influenced my understanding of the role of the school Likert 

counselor. 

17. This instrument helped to clarify the direction my school counselor(s) needs to Likert 

take. 

18. This instrument helped open dialogue with the school counselor(s) about the role Likert 

of school counselor at our school. 
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Item Text 

Response 

19. 

This instrument is an improvement compared to our former school counselor 
evaluation. 

Likert 

20. 

Which standard(s) do you feel your school counselor needs the most help 
understanding or implementing? 

Forced 

21. 

Which standard(s) do you feel you need the most help understanding? 

Forced 

22. 

What improvements would you recommend to this instrument? 

Open 

23. 

What training would you suggest to effectively implement this evaluate tool? 

Forced 

24. 

It was evident that my school counselor(s) spent time preparing for the evaluation. 

Likert 

25. 

My school counselor(s) understands what the role of a school counselor is as 
defined by the American School Counselor Association. 

Likert 

26. 

1 understand what a comprehensive school counseling program is as defined by 
the American School Counselor Association. 

Likert 

27. 

We have a comprehensive school counseling program based on the ASCA Model 
at my school. 

Likert 

28. 

1 support the role of school counselor within a comprehensive school counseling 
program. 

Likert 

29. 

1 include the school counselor(s) in the development of the school’s strategic plan. 

Likert 

30. 

A school counselor operating within a comprehensive school counseling program 
would support our school strategic plan. 

Likert 

31. 

My school counselor(s) currently impact(s) student achievement and CCRPI. 

Likert 

32. 

My counselor(s) has access to students during instructional time for classroom 
lessons, small groups and individual student planning. 

Likert 

33. 

My counselor(s) is/are afforded an opportunity to provide information and educate 
our staff on their role. 

Likert 


School Counselor COUN Perception Survey. Similarly, school counselors 
participating in the piloting of the CKES instrument were invited to participate in an 
online survey measuring perceptions regarding the CKES instrument and evaluation 
process. The survey consisted of 33 questions utilizing forced-choice, open-response, 
and Likert-type scale ratings for participants’ responses (see Table 3). The majority of 
survey items were directly aligned with those on the Administrator CKES Perception 
Survey with minor wording changes. As with the Administration survey, Likert-type scale 
responses ranged from 1 (strongly disagree) to 5 (strongly agree). 
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Table 3 

School Counselor CKES Perception Survey Items 



Item Text 

Response 

1 . 

How many years have you been a school counselor? 

Forced 

2. 

The directions on the instrument were easy to follow. 

Likert 

3. 

The format of the instrument was easy to understand. 

Likert 

4. 

The instrument was easy to use. 

Likert 

5. 

The instrument was a reasonable length. 

Likert 

6. 

The preparation time for this instrument was reasonable. 

Likert 

7. 

The terminology used in the instrument was easy to understand and follow. 

Likert 

8. 

The rating scale was fair and equitable. 

Likert 

9. 

The suggested examples of evidence provided under each standard were 
helpful. 

Likert 

10. 

If you disagree, which standard(s) could benefit from more specific examples 

Forced 

11. 

The suggested examples provided under each standard adequately allowed 
me to demonstrate my work as a school counselor. 

Likert 

12. 

The difference between the exemplary and the proficient school counselor 
was clear. 

Likert 

13. 

This instrument is helpful in defining the role of the school counselor as 
outlined by the American School Counselor Association. 

Likert 

14. 

This instrument accurately reflects my current role as a school counselor. 

Likert 

15. 

This instrument adequately reflects my leadership role in my building. 

Likert 

16. 

This instrument changed or influenced my administrator’s understanding of 
the role of the school counselor. 

Likert 

17. 

This instrument helped to clarify the direction 1 need to take in my role as 
school counselor. 

Likert 

18. 

This instrument helped open dialogue with my administrator about my role 
as a school counselor at our school. 

Likert 

19. 

This instrument is an improvement compared to our former school counselor 
evaluation. 

Likert 

20. 

Which standard(s) do you feel you need the most help understanding or 
implementing? 

Forced 

21. 

Which standard(s) do you feel your administrator needs the most help 
understanding? 

Forced 

22. 

What improvements would you recommend to this instrument? 

Open 

23. 

What kind of training on the instrument would work best for you? 

Forced 

24. 

It was evident that my administrator spent time examining my evidence to 
prepare my evaluation. 

Likert 

25. 

1 understand what the role of a school counselor is as defined by the 

American School Counselor Association. 

Likert 
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Item Text 

Response 

26. 

1 understand what a comprehensive school counseling program is as defined 
by the American School Counselor Association. 

Likert 

27. 

We have a comprehensive school counseling program based on the ASCA 
Model at my school. 

Likert 

28. 

My administrator supports the role of school counselor within a 
comprehensive school counseling program. 

Likert 

29. 

My administrator includes school counselors in the development of the 
school’s strategic plan. 

Likert 

30. 

Operating within a comprehensive school counseling program, 1 can support 
our school strategic plan. 

Likert 

31. 

In my role as school counselor, 1 currently impact(s) student achievement 
and CCRPI. 

Likert 

32. 

1 have access to students during instructional time for classroom lessons, 
small groups and individual student planning. 

Likert 

33. 

1 am afforded an opportunity to provide information and educate our staff on 
my role. 

Likert 


Analyses 

Preliminary analyses consisted of descriptive statistics and reliability indices for 
the CKES instrument dataset. Secondary analyses included item correlational analysis. 
Finally, participant survey data were reviewed and summarized using descriptive 
statistics. 

Results 

Descriptive Statistics 

CKES Instrument. In addition to reviewing mean, median, and mode, the 
normality of the distribution of scores (N = 117) was determined by reviewing the 
individual items’ histograms, skewness and kurtosis statistics for individual items, and 
items’ skewness and kurtosis statistics divided by individual standard errors (see Table 
4). Initial review of items’ histograms showed minimal concern. Review of items’ 
skewness statistics indicated no items performing outside general parameters (Field, 
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2009; Tabachnick & Fidell, 2007). However, items #1, #3, #7, and #8 all presented 
kurtosis statistics warranting further analysis. This review employed the use of kurtosis 
statistics divided by the standard error of the statistic. The action suggested data from 
items #7 and #8 performed outside the expectations of a normal distribution. 


Table 4 

CKES Item Descriptive Statistics 



N 

M 

Median 

Mode 

SD 

Skewness 

Kurtosis 

Min 

Max 

Standard 1 

117 

2.33 

2.00 

2.00 

.49 

.49 

-1.21 

1.00 

3.00 

Standard 2 

117 

2.30 

2.00 

2.00 

.55 

.05 

-.57 

1.00 

3.00 

Standard 3 

117 

2.37 

2.00 

2.00 

.50 

.35 

-1.38 

1.00 

3.00 

Standard 4 

117 

2.23 

2.00 

2.00 

.44 

.99 

-.17 

1.00 

3.00 

Standard 5 

117 

2.17 

2.00 

2.00 

.50 

.34 

.46 

1.00 

3.00 

Standard 6 

117 

2.18 

2.00 

2.00 

.60 

-.08 

-.33 

1.00 

3.00 

Standard 7 

118 

2.47 

2.00 

2.00 

.50 

.14 

-2.02 

2.00 

3.00 

Standard 8 

117 

2.35 

2.00 

2.00 

.48 

.64 

-1.62 

2.00 

3.00 

Standard 9 

118 

2.43 

2.00 

2.00 

.56 

-.32 

-.86 

1.00 

3.00 

Standard 

10 

117 

2.35 

2.00 

2.00 

.53 

.10 

-.91 

1.00 

3.00 

Overall 

Rating 

117 

23.21 

22.00 

20.00 

3.93 

.32 

-.66 

12.00 

30.00 


Reliability of the CKES instrument was assessed reviewing Cronbach’s Alpha. 
The Cronbach Alpha statistic for the CKES was a = 0.920. Consideration of removal of 
any of the 10 individual items yielded no increase in alpha. Next, inter-item correlations 
between the 10 items on the CKES were reviewed (see Table 5). Correlations ranged 
from 0.28 to 0.82 with a mean average 0.54. The weakest correlation (0.28) was found 
between items #6 (data evaluation) and #9 (professionalism). The strongest correlations 
were found in relationships between the overall rating and individual items (i.e., 0.71 < r 
< 0.82). Individual items #1, #2, and #10 demonstrated concerning correlations to the 
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overall rating score with r= 0.82. Other than these instances, item correlations were 
found to be within acceptable parameters (Field, 2009; Pett, Lackey, & Sullivan, 2003). 


Table 5 

CKES Item Correlations 



2 

3 

4 

5 

6 

7 

8 

9 

10 

Overall 

Rating 

Standard 1 

.65* 

.51* 

.59* 

.61* 

.56* 

.56* 

.60* 

.57* 

.64* 

.82* 

Standard 2 


.67* 

.57* 

.67* 

.63* 

.47* 

.59* 

.45* 

.56* 

.82* 

Standard 3 



.47* 

.54* 

.53* 

.52* 

.64* 

.47* 

.61* 

.77* 

Standard 4 




.56* 

.43* 

.45* 

.43* 

.46* 

.50* 

.71* 

Standard 5 





.68* 

.51* 

.47* 

.42* 

.59* 

.79* 

Standard 6 






.36* 

.47* 

.28* 

.51* 

.72* 

Standard 7 







.47* 

.69* 

.68* 

.74* 

Standard 8 








.49* 

.53* 

.74* 

Standard 9 









.68* 

.72* 

Standard 10 










.82* 


Note. * denotes significant at p < .01 


Administrator CKES Perception Survey. Of the 33 survey questions presented 
to participating administrators (N = 24) 16 specifically addressed CKES reception, use, 
and performance. The remaining 17 questions were divided between determining site 
specifics (e.g., Q1: How many years have you supervised the school counselor(s)?) and 
assessing participant awareness of comprehensive school counseling and school 
counselor role (e.g., Q26: / understand what a comprehensive school counseling 
program is as defined by the American School Counselor Association.). Table 6 
presents descriptive statistics for the 16 CKES -related items. 

All items performed similar with median and mode scores of 4.00 (“agree” on the 
survey Likert-type scale). Mean average scores ranged between 4.13 and 4.38 for 
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nearly all items with only #16 (This instrument changed or influenced my understanding 
of the role of the school counselor.) deviating from this pattern reporting a mean 
average score of 3.46. 

School Counselor CKES Perception Survey. Similar to the administrator 
perception survey, participating school counselors (N = 40) responded to 16 questions 
addressing CKES reception, use, and performance with the remaining 17 dedicated to 
site specifics (e.g., Q1: How many years have you been a school counselor?) and 
knowledge of comprehensive school counseling and school counselor role (e.g., Q26: / 
understand what a comprehensive school counseling program is as defined by the 
American School Counselor Association.). Table 6 presents descriptive statistics for the 
16 CKES-related items. 



Table 6 

CKES Perception Survey: Administrators and School Counselors 
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Administrator Responses School Counselor Responses 


Survey Item 

n 

M 

Median 

Mode 

Fav% 

Unfav% 

n 

M 

Median 

Mode 

Fav% 

Unfav% 

2. Directions easy to follow 

24 

4.13 

4.00 

4.00 

91.67% 

4.17% 

40 

3.78 

4.00 

4.00 

75.00% 

7.50% 

3. Instrument easy to understand 

24 

4.21 

4.00 

4.00 

95.83% 

0.00% 

40 

3.78 

4.00 

4.00 

72.50% 

5.00% 

4. Instrument easy to use 

24 

4.13 

4.00 

4.00 

91.67% 

8.33% 

40 

3.63 

4.00 

4.00 

65.00% 

7.50% 

5. Instrument reasonable length 

24 

4.29 

4.00 

4.00 

100.00% 

0.00% 

40 

3.58 

4.00 

4.00 

67.50% 

15.00% 

7. Terminology easy to understand 

24 

4.17 

4.00 

4.00 

91.67% 

4.17% 

40 

3.80 

4.00 

4.00 

80.00% 

7.50% 

8. Rating scale fair and equitable 

24 

4.38 

4.00 

4.00 

95.83% 

0.00% 

40 

3.78 

4.00 

4.00 

72.50% 

7.50% 

9. Examples helpful 

24 

4.25 

4.00 

4.00 

91.67% 

4.17% 

40 

3.88 

4.00 

4.00 

70.00% 

5.00% 

11. Examples allowed me to evaluate 

24 

4.25 

4.00 

4.00 

95.83% 

4.17% 

40 

4.43 

4.00 

4.00 

75.00% 

10.00% 

12. Distinguishing exemplary and proficient 
was clear 

24 

4.21 

4.00 

4.00 

91.67% 

4.17% 

40 

3.65 

4.00 

4.00 

70.00% 

20.00% 

13. Instrument helpful defining role of 
school counselor outlined by ASCA 

24 

4.17 

4.00 

4.00 

87.50% 

0.00% 

40 

4.05 

4.00 

4.00 

82.50% 

2.50% 

14. Instrument reflects role of school 
counselor in my school 

24 

4.25 

4.00 

4.00 

87.50% 

0.00% 

40 

3.58 

4.00 

4.00 

62.50% 

17.50% 

15. Instrument reflects leadership role of 
school counselor in my building 

24 

4.21 

4.00 

4.00 

87.50% 

0.00% 

40 

3.63 

4.00 

4.00 

65.00% 

12.50% 

16. Instrument changed understanding of 
role of school counselor 

24 

3.46 

4.00 

4.00 

58.33% 

33.33% 

40 

3.28 

3.00 

3.00 

37.50% 

15.00% 

17. Instrument helped clarify direction 
school counselor(s) needs to take 

24 

4.13 

4.00 

4.00 

83.33% 

0.00% 

40 

3.85 

4.00 

4.00 

77.50% 

10.00% 

18. Instrument opened dialogue with 
school counselor(s) about role at our 
school 

24 

4.13 

4.00 

4.00 

91.67% 

4.17% 

40 

3.40 

4.00 

4.00 

55.00% 

20.00% 

19. Instrument an improvement compared 
to former school counselor evaluation 

24 

4.21 

4.00 

4.00 

83.33% 

4.17% 

40 

3.40 

3.00 

3.00 

47.50% 

22.50% 


Note. Range for all items was 1-5. “Fav%” reflects aggregate percentage “Agree” and “Strongly Agree.” “Unfav%” reflects aggregate percentage “Disagree’ 
and “Strongly Disagree.” 






Data from participating school counselors presented slightly more variance than 
administrators. Similar to the administrator survey results, data for most items reported 
median and mode scores of 4.00 with mean average scores ranging from 3.58 to 3.88. 
Both items #16 and #19 reported median and mode scores of 3.00 with mean averages 
of 3.28 and 3.40 (respectively). 

Discussion 

CKES Instrument 

Overall, the CKES instrument appeared to perform reasonably well in evaluating 
school counselors’ performance with only two items warranting concern (i.e., items 7 
and 8). The instrument demonstrated good reliability and most item correlations were 
within expected parameters. However, some strong correlations were observed 
including items 5 and 6, items 7, 9, and 10, and the summative “overall rating.” 

Items 5 (data collection) used the prompt, “The professional school counselor 
uses a variety of strategies and instruments to collect student data in order to guide 
appropriate counseling interventions and programs” to guide rater Likert scaling, while 
item 6 (data evaluation) stated, “The professional school counselor evaluates student 
data and the effectiveness of the counseling core curriculum, small groups, and closing- 
the-gap data, and shares program results with stakeholders.” In the current study, the 
items demonstrated a correlation of r = .68. Both items address the use data (i.e., 
collection and evaluation), and that such use informs action to be taken (i.e., the phrase, 
“...in order to...”). Additionally, as both items fall within the ASCA National Model (2012) 
domains of accountability system and management system, it seems plausible these 
items might demonstrate overlap in their measurement of latent construct(s). 
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Not quite as straight-forward, item 7 strongly correlated with items 9 and 10, 
demonstrating values of r= .69 and r= .68 (respectively). Item 7 presented the prompt, 
“The professional school counselor promotes a safe, positive learning environment 
which is inclusive of ALL students (including but not limited to race, color, religion, 
gender, national origin or disability) and advocates for student needs in order to reach 
their educational goals.” Item 9 read, “The professional school counselor exhibits a 
commitment to professional ethics and the mission, vision and beliefs of the school 
counseling program and participates in professional growth opportunities.” Item 10 
stated, “The professional school counselor communicates effectively with students, 
parents/guardians, district and school personnel, and other stakeholders in a way that 
enhances student learning and improves the comprehensive school counseling 
program.” While both items 7 and 9 addressed the Foundation domain of the ASCA 
National Model (2012), item 10 focused on the delivery system and management 
system. One possible factor may the shared focus on student learning evident in the 
text of both items 7 and 10. 

Finally, the summative item “overall rating” demonstrated the strongest 
correlations of all items on the CKES with values ranging from r= .71 to r= .82. This 
result may not be surprising considering the item was an aggregation of the other 10 
items. However, two other considerations are worth noting. First, with many 
administrators still unfamiliar with school counselor role and responsibilities (Morris & 
Slaten, 2014) there may be reluctance to assess evident deficiencies resulting in 
inflated scores (Anderson, 1994). Second, guiding language for scoring the “overall 
rating” item may have unintentionally encouraged raters to reconsider previous item 
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ratings. Each level of performance (i.e., exemplary, proficient, needs development, and 
ineffective) was accompanied with guiding criteria directing rater scoring. For example, 
to achieve a rating of “exemplary,” participating school counselors must have received a 
total of 27-30 points and no “needs development” or “ineffective” scores (i.e., Likert- 
rating 1 or 0) on any of the 10 items. Achieving a summative rating of “proficient” carried 
similar criteria as well. For example, if a school counselor had accumulated 26 points 
but received a “needs development” (Likert-rating 1) on one of the 10 items, the rater 
would be required to assign a summative rating of “needs development.” 

Participants’ Perceptions 

Overall participating school counselors and administrators seemed pleased with 
the performance of the CKES evaluation instrument. Reviewing the data, Likert- 
responses “strongly agree” and “agree” were aggregated into the variable “Fav%” while 
“strongly disagree” and “disagree” were combined to form “Unfav%.” These variables 
reflected the percentage of favorable or unfavorable ratings excluding “neutral” 
responses. As a result, readers will note that for some items (see Table 6) the sum of 
Fav% and Unfav% will not equal 100%. 

Reviewing the Fav% and Unfav% variables for each item, the majority of 
respondents indicated favorable perceptions of the CKES instrument and its 
implementation as an evaluation tool for school counselors with the exception of item 
16. Additionally, for every item of the survey administrators responded more favorably 
than school counselors. This difference between administrator and school counselor 
responses ranged from 5.0 (item 13, “Instrument is helpful defining the role of the 
school counselor outlined by ASCA”) to 36.7 points (item 18, “Instrument opened 
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dialogue with school counselors about role at our school”). Recognizing the potential 
impact evaluation processes can have when advocating for school counselor 
program/role with administrators (Morris & Slaten, 2014), this trend is indeed 
encouraging. 

On the school counselor perception survey, item 16 read, “This instrument 
changed or influenced my administrator’s understanding of the role of the school 
counselor,” while the administrator version of the prompt read, “This instrument 
changed or influenced my understanding of the role of the school counselor.” Item 16 
demonstrated the lowest mean scores for both administrator (3.46) and school 
counselor (3.28) respondents. Aggregation of item rating scores on the administrator 
survey still yielded a majority responding favorably (58.33%). However, for participating 
school counselors only 37.5% reported favorable perceptions while 15.0% reported 
unfavorable. Overall this finding would appear to be contrary to literature suggesting 
that a primary function of school counselor evaluation is facilitating administrator 
awareness/education of the school counselor role (Anderson, 1994; Morris & Slaten, 
2014; Studer & Sommers, 2000). While editing of the item wording might be utilized to 
provide more focus to this item, it would be beneficial to continue piloting to increase the 
sample prior to making changes to the protocol. 

Finally, review of the highest ratings in the dataset revealed a practical focus in 
how respondents perceived the CKES instrument. For administrators, item 5 
(Instrument reasonable length) demonstrated a Fav% value of 100% while item 8 
(Rating scale fair and equitable) demonstrated a mean of 4.38. Similarly, school 
counselors responding to item 11 (Examples allow me to evaluate) demonstrated a 
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mean of 4.43 and a Fav% value of 82.5% for item 13 (Helpful defining role of school 
counselor outlined by ASCA). These findings suggest that among the potential themes 
alluded to within the survey (e.g., evaluation processes, school direction, school 
counselor role), the logistical aspects of the CKES instrument were found to be most 
favorable (ergo most important) to participants. 

Limitations 

A considerable limitation of this study was sampling, in terms of both procedure 
and size. While the convenience sampling utilized in this study facilitated the pilot 
process and allowed the study to mirror statewide practitioner trends (i.e., state-location 
and site-level), it lacked the statistical rigor of random sampling. Additionally, while the 
CKES participation sample size (N = 117) might be considered moderate to good, the 
administrator and school counselor perception samples sizes (N = 24 and N = 40, 
respectively) were both notably lower. Larger sampling of the perception groups would 
further validate initial findings while larger CKES participation sampling would facilitate 
additional statistical analyses. 

Similarly, the cumulative collaborative endeavor itself was not without limitations. 
Most noticeably, the challenge of utilizing volunteer resources (e.g., state association 
leaders and members) while attempting to achieve outcomes determined by state-level 
policy makers proved formidable. This challenge is further elaborated momentarily after 
first addressing research recommendations. 
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Recommendations 

Research 

Larger sample sizes, as referenced in the limitations section, would specifically 
facilitate exploratory factor analysis of the CKES instrument as well as correlational 
analyses with both teacher and administrator versions of the instrument (i.e., TKES and 
LKES). An exploratory factor analysis of the CKES instrument would identify latent 
construct(s) and determine where individual items aligned with such construct(s). Such 
analysis might shed further light on some of the stronger inter-item correlations 
highlighted in the results section (e.g., items #1, #2, and #10). Additionally, exploratory 
factor analysis would yield a more complete picture of whether the CKES is measuring 
a single construct such as “school counselor efficacy,” or multiple constructs such as 
various domains of the ASCA National Model (2012). 

Similar to other collaborative approaches to school counselor evaluation, created 
school counselor instruments have relied heavily upon pre-existing teacher and/or 
administrator instruments. These instruments often have political capital/momentum 
within the state facilitating school counseling leaders to advocate for role definition 
through customization of protocol items. Correlational analyses between the CKES and 
“original” teacher and administrator versions of the instrument would provide insight into 
the relationship between instruments and how much overlap may or may not be 
occurring. As previously mentioned in regards to exploratory factor analysis, such 
overlap may be quite pronounced as the result of unidimensional constructs (e.g., 
efficacy) or may be more minimal if instruments explicitly focus on aspects of 
role/position. 
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It may seem obvious that a primary recommendation is collaboration with the 
state department of education and state-level policy makers. Perhaps more instructive 
is to discuss the nature of such collaboration. The authors return to an ecological school 
counseling systems perspective (McMahon, Mason, Daluga-Guenther, & Ruiz, 2014) 
highlight the nature of collaboration via specific roles fulfilled by individual system 
agents (i.e., state department of education, state school counselor association, and 
university faculty). These roles include implementation, instrumentation, and 
awareness. The authors illustrate these roles by highlighting parallels between the 
current study and a similar study over twenty years ago. 

Systemic Collaboration 

During literature review for the current study the authors reviewed a paper 
presented to the American Educational Research Association (Anderson, 1994) 
outlining the development, piloting and implementation of a School Counselor 
Evaluation Program (SCEP) begun in 1989 within the same state as the current study. 
Though separated by over 20 years, the authors noticed numerous similarities between 
the two studies. Both studies illustrated systemic environments where a climate of 
accountability resulted in the state allocating millions of dollars towards evaluation 
programs and processes. Similarly, both studies used teacher and administrator 
instruments as templates, building/directing political momentum surrounding teacher 
evaluation towards school counseling. At first glance, such direct parallels may be 
disheartening to school counseling advocates as it would appear no progress towards 
effective school counselor evaluation was made within the macro-system over the 
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course of 20 years. However, differences between the two studies in regards to 
implementation, instrumentation, and awareness may suggest a more hopeful picture. 

Implementation: Department of education collaboration. Despite member 
enthusiasm and strong volunteering, few state associations possess the resources to 
implement state-wide evaluation processes for school counselors. Thus an important 
component to successful school counselor evaluation is collaboration with the state 
department of education (DOE) where DOE resources are made available. These 
resources extend beyond finances to include communication, administrative authority, 
and relational capital. Additionally, DOE collaboration may provide critical framing for 
how evaluation data is intended to be utilized and reviewed. Using a dichotomous 
pass/fail grading scale, the SCEP was designed to be used as a tool to facilitate 
conversation between administration and school counselors as to how to direct 
professional growth activities (Anderson, 1994). Consequently, pilot study results 
indicated approximately 99% of all participating school counselors were performing at 
“satisfactory” level. While this statistic might seem encouraging, many state-level 
stakeholders found the data troubling as it did not provide as complete a picture as 
desired. 

In contrast, the CKES evaluation was designed to provide a data picture aligned 
with the already approved teacher and administrator protocols. The same desire to 
stimulate professional growth conversations were a part of the CKES process, however 
resources geared towards this endeavor (e.g., descriptions of role and responsibilities, 
appropriate/inappropriate duties, examples of each rating level) were included as 
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supplementary to the CKES instrument itself. In this way the state association hoped to 
accomplish both goals of alignment with DOE data needs and professional advocacy. 

Instrumentation: University Collaboration. In both the current study as well as 
the Anderson (1996) study, substantial research resources were dedicated towards 
instrument construction for teacher evaluation with relatively little to none provided for 
school counseling. Anderson (1996) shows the tenuous nature of such funding in that 
while school counseling initially received similar research support, when budget cuts 
became necessary smaller population programs were quickly eliminated to maintain 
larger programs (i.e., 2,000 school counselors versus 66,000 teachers). In the present 
study, DOE leadership (i.e., state superintendent) was supportive of the creation of a 
school counselor evaluation tool, but clearly articulated the lack of staff, financial 
monies, and research-support available. In both studies, the state school counseling 
association (SCA) took initiative to pursue instrument creation on their own. 

SCA leadership and members provide crucial insight into the tasks, duties, 
expectations, and challenges facing practicing school counselors in the PK-12 arena. 
Such information is necessary for creating instrument items/statements that accurately 
reflect practitioner working environments. However, few SCAs have research staff and 
support services necessary for in-depth statistical analysis of created instruments. This 
highlights an important aspect of successful collaboration partnering university faculty 
can bring. University faculty can provide SCAs with staff, support, and technical 
assistance exploring the psychometric properties of created evaluation instruments. 
Where preliminary program evaluation analyses might suffice for district-level review, 
the continually increasing focus on accountability requires additional investigations 
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exploring instrument construction, validity, and reliability if SCAs are to effectively 
advocate at state and national levels. Even when state DOE resources are made 
available, university partnerships can augment those services and maintain continuity 
throughout fiscal fluctuations. 

Awareness: School Counseling Association Collaboration. As highlighted in 
reviewing these paralleling studies, feedback loops perform a crucial function of 
ecological systems whether developing through intentional, structured activities or as 
spontaneous behaviors organically emerging from system members (McMahon, Mason, 
Daluga-Guenther, & Ruiz, 2014). Capitalizing on such feedback in order to accomplish 
healthy systemic development requires both awareness and memory. State school 
counseling associations (SCAs) can perform both of these roles thus fostering 
successful collaboration between all system stakeholders. 

Systemic awareness might be most readily illustrated by the presence of SCA 
members/teams pursuing advocacy within state-level politics. SCAs are able to monitor 
multiple initiatives, proposals, and policies in ways which exceed dedicated SCA 
personnel. The resulting picture SCAs receive is systemically-framed or “scaffolded” as 
information is provided from the perspective of individual (i.e., SCA member) as well as 
from various subsystems (e.g., district, region, etc.). This affords SCAs the ability to 
craft responses/actions that carry the systemic “weight” of a larger system agent, while 
incorporating feedback from multiple members and levels within the system. Regarding 
school counselor evaluation, this means SCAs may be able to align with state-level 
initiatives (identified from individual members and sub-systems) long before receiving 
any official communication from state-level policy makers. In regards to school 
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counselor evaluation, individual SCA members may be able to encourage legislative 
representatives to support SCA initiatives without the presence of formally introduced 
bills. Similarly, in response to SCA calls for support, individual members may be able to 
communicate regional/district techniques and resources. 

Building off of awareness, systemic memory might be considered the 
maintenance of a historical perspective of agents’ experience within the system. 
Whether recent occurrence or long ago, such a perspective can be highly valuable in 
legislative arenas when political changes demand “new” directions departing from 
previous systemic initiatives regardless of their merit. Once again, SCAs afford a 
“scaffolded” or nested perspective as individual members retain and utilize systemic 
memories that intermingle and inform the SCA collective. For state associations 
espousing a policy governance model (Carver & Carver, 2006), this sort of nesting 
allows the SCA to construct, retain, and act on its own meso-level-agent memories. An 
example of systemic memory may be gleaned by returning again to review of the 
current study and Anderson (1994). 

In the Anderson (1994) study, instrument preparation and development took 
approximately three years before piloting. Sadly, even with this development the 
purpose and function of the instrument did not align with policy-makers expectations or 
agendas. In the current study, instrument preparation took significantly less time and 
mirrored both teacher and administrator instruments in hopes of conveying information 
in the expected formats. Maintaining these memories (both individually and collectively) 
will help the SCA navigate continually changing state-level politics, momentum, and 
initiatives. Similarly, recognizing limited DOE resources, the SCA in the current study 
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acted quickly to volunteer SCA resources and personnel. Retaining this experience 
within SCA memory benefits future DOE collaboration not only in terms of logistics (e.g., 
instrument creation, distribution of protocols to pilot districts, state-wide communication) 
but also in fostering healthy dialog between agencies (e.g., points of contact made, 
networks established, example of “successful” collaboration to reference as collateral). 

Conclusion 

This study set out to create and pilot the (CKES) school counselor evaluation 
instrument. The project involved collaboration between the state Department of 
Education (DOE), the state School Counselor Association (SCA), and local university 
faculty. Results suggest that overall the CKES instrument functioned well and was 
perceived favorably by participating school counselors and administrators. 

Specifically, the straight-forward logistics and practical focus of the CKES 
appeared to be the most valuable aspects of the instrument. Interestingly, 
administrators perceived the CKES even more favorably than school counselors. This 
garnering of administrator support is particularly encouraging considering the 
prevalence of administrator unfamiliarity with school counselor role, function, and 
training (Morris & Slaten, 2014). 

The evaluation of school counselors presents a complex endeavor situated within 
multiple competing agendas. Creation and implementation of instruments such as the 
CKES depends upon healthy collaboration between system stakeholders (i.e., DOE, 
SCA, and university). Only when such collaboration is paired with a statistically sound 
instrument can meaningful gains in school counselor evaluation, accountability, and 
advocacy be achieved. 
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